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The Tell-Tale Data: Virtual Whispering and Final Student Grades 

Abstract 

Online classroom management issues pose new problems for the online instructor and 
pose seductive communicative options for students. This exploratory group of studies 
examined Blackboard/WEBCT™ data as collected for the course designer of an online 
course as possible indicators of “whispering” or backchanneling between students with 
potential impacts on course outcomes (grades). Three studies examined possible 
significant relationships between WEBCT data (HITS, READ, POSTED, and MAIL) 
with student final grades. Studies varied the independent variables of course content, 
instructor, length of course, and enrollments. Enrollments in studied courses ranged from 
nine students to 17. Correlational matrices and cluster analyses were performed using 
SPSS ver. 13. Findings included a strong significant relationship between HITS and 
student final grades across all studies, while MAIL (student- student interactions) failed to 
correlate significantly with final grades except in one lower-level course. Cluster analyses 
identified from two to three clustered groups within each study. Clusters were determined 
by hierarchically organizing WEBCT™ data as follows: HITS, READ, POSTED, 

MAIL, and GRADE (ltr). Analyses of clusters supported the relationships noted between 
HITS and student final grades, but varied on the value of MAIL data relationships. 
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The Tell-Tale Data: Virtual Whispering in Online Courses 
Introduction 

A recent review of Web-based protocols for students studying in an online 
environment revealed the following instruction: 

Side Conversations in class - Also, like a traditional classroom, there can be no 
whispering in the back of the room. You may only raise your hand or post a 
question. 

(http://www.infosec.imu.edu/courses/CommonWebPages/ChatProtocol.html , p. 3) 
This instruction is a stretch of the imagination for anyone who has taught in both the 
traditional and online format. How can a student raise a hand? How can an instructor tell 
if anyone is whispering in the back of the room? And, even more importantly, what 
difference does the potential whispering in the back of the room make for the student, the 
student’s grade, or for the instructor? This study examines the value of WEBCT data 
from online courses as an indicator of the impact of virtual whispering on learning 
outcomes. First, terms are defined reflecting the use of communication terms in the 
virtual WEBCT domain. Second, extant research is reviewed to see what, if anything, has 
contributed to an understanding of this phenomenon — whispering in the back of the 
virtual classroom. Finally, the analysis of data in three studies collected from online 
classes taught in WEBCT is offered, followed by a discussion of the possible 
interpretations of these data for the online instructor. 

Terms Defined 

Computer-mediated communication frames the interactions with the virtually- 
taught course. Two categories of interaction generally comprise web-based learning - 
synchronous (real-time interactions) and asynchronous (time and place independent) 
communication. Often synchronous communication is difficult to impossible if students 
and professor are geographically diverse. Thus, this study focuses on the value and 
contribution of asynchronous communication between participants in virtual learning. 
Three forms of interaction are likely in this category of communication - 1) 
communication through course web pages (the least interactive approach), 2) postings to 
a discussion forum, and 3) e-mail messages through the course shell (Lavooy and 
Newlin, 2003). 

Blackboard/ WEBCT™ is a commercial product for online course delivery, 
usually employed by colleges and universities. The course environment in Blackboard / 
WEBCT™ is designed by the instructor to include any of the following options: syllabus 
information equivalent to a hard copy syllabus, course content materials, discussion 
folders, assignment instructions with submission links, group projects, and e-mail within 
the course shell. Tracking data are collected through WEBCT™ so that access to specific 
course sections and materials are logged for each individual student. A history of the data 
is available to the instructor throughout the semester as well as at the conclusion of the 
course. 

The following data are collected per individual student (Blackboard / WEBCT™ 
Glossary, http://socrates.berkeley.edu/~fmb/articles/webct-glossary.html): 

Homepage visits Organizer page visits 

Content Page visits Assignment visits 

Quiz visits (if used) Calendar visits 

Mail events My Grades visits 
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Articles read Original Posts 

Follow-up Posts Last login 

Last page visited Total number of accesses 

Most of the collected data provides an interesting picture of overall student interactions, 
but few studies have examined the relationships between the data and student outcomes. 
Blackboard / WEBCT™ is usually regarded as a “teaching tool”(Brown Daily Herald, 
MarchlO, 2005), while the tracking facilities are often ignored or employed only to check 
on students whose participation has been questioned. 

E-mail within a web-based course takes on two roles. First, students can e-mail 
an instructor within the course, and the instructor is alerted to the email by a letter icon or 
an asterisk by MAIL in the Course Tools each time the instructor logs in. These e-mails 
appear only in the course shell and are generally regarded as course- specific interaction. 
Second, students can e-mail each other within the course. These e-mails may occur 
within a discussion forum in the course (student-to-student or student-to-instructor) or 
privately between students without the instructor’s knowledge of content. Of particular 
interest to this study is the latter, the private student-to-student e-mails with unknown and 
unmonitored content. 

Backchanneling or “whispering in the digital classroom” (Cogdill, Fanderclai, 
Kilbom & Williams 2001) is defined as private communication visible only to the sender 
and receiver, in this case two students, creating the “whispering” property. The Web 
Dictionary of Cybernetics and Systems (2000) further defines this phenomenon as 
“communication which travels through informal rather than formal channels.” Of 
particular interest for this study and its focus are the factors relating to private nonvisible 
content and the informal channel. Backchanneling could feasibly be course-related, 
participation-enabling, tangential, or process-related interaction between students in an 
effort to assist each other in successful completion of the course assignments. However, 
the possibility of personal interaction that is off-topic or destructive to the learning 
environment or instructor exists as well. If we reflect on our understanding of 
“whispering” in a traditional classroom, this possibility is logical. Yet, studying the 
backchannel for this category of interaction is difficult, considering the likelihood that 
announcing a study of backchannel interactions will change the tenor and content of the 
exchanged e-mails. And yet, the question remains - what impact does this personal 
backchannel have on the course and the student’s outcome? Is this form of 
communication contributing to student outcomes, and is the contribution significant 
statistically? 

Most of the literature to date has provided little in response to this question. As 
mentioned already, Cogdill et al (2001) provided a definitional and ethical discussion of 
backchanneling and its presence in the online classroom. The value of the backchannel 
and its categories is clearly helpful for online instructors, but the mystery of the content 
and the persistence of backchannel communication raise the specter of concerns without 
obvious technological or instructor-initiated solutions (Cogdill et al, 2001). 

Lavooy and Newlin (2003) suggested in their study that computer-mediated 
communication and web instruction fostered increased student- student communication as 
well as the essential instructor-student interaction. Logically, interactivity in an online 
course is critical to instantiating the course environment. Additionally, students who 
engage and interact often reduce the isolative feelings that may be dreaded by students in 
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online courses (Lavooy and Newlin, 2003, p. 3). These student-to-student interactions 
may indeed take the form of forum postings though, removing the concerns inherent in 
personal, off-topic backchanneling (Wang and Newlin, 2000). 

Wang and Newlin (2000) also provided substantive input on student 
characteristics and student success in online courses. Given the measures of online course 
activity already mentioned, the Wang-Newlin study found that successful online students 
(as measured by final grades) exhibited a greater internal locus of control, higher need for 
cognition, and interacted more through forum postings and readings than their less 
successful counterparts. Reynolds, Cox, Rice and Uddin (2001) extended the value of 
Blackboard / WEBCT™ data by collecting site and page hits for online therapeutic- 
course participants for a year, though their efforts only sought to describe students’ usage 
over time without examining relationships between usage and final outcomes. As well, 
the component for private backchanneling is ignored in this study. 

Merceron and Yacef (2004) examined student data captured from a web-based 
tutoring tool with a goal of improving teaching while better understanding student usage 
of the tool. Marra (2006) most recently examined research methods that would help 
instructors understand and analyze student participation and content in online discussion 
forums. Different coding schema as well as numerical counts for postings and readings 
offered insights into these student interactions, but once again content is visible and the 
communication is deemed public, not private. 

The current study examines the meaning of the Blackboard / WEBCT™ data for 
online classes taught in the Department of Technical Communication and Media 
Production at an urban college in a large city in the Rocky Mountain region. The 
questions driving this study are as follows: 

• What significant relationships exist among the Blackboard / WEBCT™ 
data for an online course that reflect student outcomes as defined by 
final grade? 

• Without viewing or analyzing content of backchannel interactions 
between students, is productive (course-or content-related) or off-topic 
(personal or negative) engagement capable of being logically inferred 
from the online course data? 

• Which Blackboard / WEBCT™ data contribute the strongest significant 
relationships with student outcomes as measured by final grade? 

• Finally, are relational hypotheses possible regarding student outcomes 
based upon multiple-course data as gathered through Blackboard / 
WEBCT™ software? 



METHODS 
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Three studies were conducted to examine these research questions when 
considering frequently-varied course components such as length of the course, different 
instructors, and same instructor/different courses for any impact on data relationships in 
Blackboard / WEBCT™. Two comparative studies were conducted using one instructor 
teaching two simultaneously offered courses, different content. One study reflects the 
shorter course term (eight weeks) with the same instructor offering different course 
content. And finally, the last study examined one instructor in two sections of one course 
online to see what relationships emerged reflecting student outcomes. 

The instructors were all faculty within the Department of Technical 
Communication and Media Production at a large urban college in the Colorado. All 
instructors had taught online courses for three to five years as well as having taught the 
same course material in courses on campus. Enrollments in the courses ranged from nine 
students to 18 students (the usual cap for online courses at this college). Only students 
who completed the courses and received a grade were included in the statistical analyses. 
Demographics for the students were not collected as this exploratory group of studies 
focused primarily on the Blackboard / WEBCT™ data and any possible relationships to 
student outcomes (grades). 

At the conclusion of the semester or the course (for eight week courses), each 
instructor for the individual courses collected data from the Blackboard / WEBCT™ 
course shell. Table 1 below shows the data that were collected from the MANAGE 
COURSE link available in Blackboard / WEBCT™ for course designers (aka 
instructors). 



Table 1. Blackboard / WEBCT™ Data collected for each online course in each of three exploratory 
comparisons 



Student course # - # 


Definition of this variable 
Course #s include four numerals 
and each student was assigned a 
number in sequence 


Comments 

No identifying information about 
the student was included in the 
analysis 


HITS 


Number of pages visited within 
the course, i.e.. Homepage, 
organizer, assignments, etc. 


A total number of HITS for the 
semester was recorded for the 
analysis. 


READ 


Number of articles read within 
the course 




POSTED 


Number of postings to discussion 
forums 




FOLLUP 


Number of follow-up postings to 
discussion comments in forums 




MAIL 

GRADE 


Number of emails sent within the 
course 

Letter grade recorded by the 
instructor for the student’s 
semester work 


This number includes student-to- 
student emails AND student-to- 
teacher emails. 

The standard unit value for letter 
grades (A = 4, B = 3) were used 
in the analyses. 


GRADEPC 


The % grade for the student’s 
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work in the course 



Once data were collected for each study, descriptive analyses were performed using 
SPSS (Statistical Package for the Social Sciences, ver. 13) and then correlation matrices 
were examined for each course within each study. Finally, cluster analyses were 
performed using SPSS to “data-mine” for specific related groups of individuals within 
courses using the Blackboard / WEBCT™ data for HITS, READ, POSTED, GRDPRCT, 
and MAIL. Hierarchical cluster analyses were performed first to examine the average 
linkages between groups using the visual dendogram. Following this analysis, quick 
clusters were then examined for cluster membership and distances between clusters. 



RESULTS 

Study 1 - Comparing Two Courses (different content) taught by a single instructor, 16 
weeks long 

For this study, two sets of two courses taught by single instructors were 
compared. Blackboard / WEBCT™ data for each course were gathered. Professors Alpha 
and Beta taught both a sophomore level course (COM 2610) and an upper division course 
(COM3670) taken by juniors and seniors. Courses taught by Professor Alpha were 
required courses for majors in the department, though the lower level course is 
considered a “service course” for other technical majors in other disciplines. Courses 
taught by Professor Beta were also one lower division course (COM 2460) though the 
subject matter related to Media Graphics and Presentations, while the second course was 
an upper division course (COM 3470). Descriptive data for the two-course comparisons 
are provided in Tables 2 and 3 below. Calculated statistics are rounded up to two places 
beyond the decimal point. 

Table 2. Descriptive Statistics of Blackboard / WEBCT ™ variables for Professor Alpha Courses 



COM 2610 (n 


= 14) and COM 3670 (n = 


16). 














Minimum 


Maximum 


Mean 


SD 




2610 


3670 


2610 


3670 


2610 


3670 


2610 


3670 


HITS 


120 


185 


1329 


866 


547.71 


498.81 


336.46 


225.14 


READ 


2 


0 


467 


182 


137.79 


109.63 


152.30 


74.55 


POSTED 


0 


0 


53 


34 


18.29 


6.63 


19.25 


9.04 


MAIL 


6 


13 


88 


122 


46.36 


58 


27.48 


31.04 


FOLLUP 


0 


0 


53 


34 


18.36 


6.44 


4.67 


2.50 


GRADE 


0 


0 


4 


4 


2.93 


2.56 


1.21 


1.41 



Table 3. Descriptive Statistics of Blackboard / WEBCT™ variables for Professor Beta Courses 
COM 2460 (n = 16) and COM347Q(n = 17). 





Minimum 


Maximum 


Mean 


SD 




2460 


3470 


2460 


3470 


2460 


3470 


2460 


3470 


HITS 


18 


173 


917 


1779 


316.56 


815.35 


232.31 


413.24 


READ 


0 


39 


181 


511 


84.88 


266.76 


63.18 


174.94 


POSTED 


0 


5 


34 


70 


8.81 


25.59 


9.26 


19.82 


MAIL 


0 


16 


120 


185 


31.19 


62.88 


27.73 


42.07 


FOLLUP 


0 


1 


33 


60 


7.63 


17.65 


8.57 


18.57 


GRADE 


0 


0 


4 


4 


2.75 


2.94 


1.91 


1.48 
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Apparent from these tables, Professor Alpha’s students in 2610 engaged in the course 
more frequently to read course materials, post to the discussion forums, and respond to 
comments in the discussions. Professor Alpha’s students in 3670 engaged in the course 
less frequently when compared to the 2610 students, but mailed more personal 
communications as noted by the maximum number of mails and the mean value across all 
students. Professor Beta’s students in 2460 and 3470 reversed this trend. Students in 3470 
engaged frequently in the course by reading more course materials, posted more to the 
discussion forums, and responded more to comments in the forums than the students in 
2460. Aggregate grades for all students in both course comparisons vary little on the 
range and the measures of central tendency. 

The second phase of statistical analysis involved examining the correlation 
matrices for both compared courses. The question driving this analysis sought to uncover 
significant relationships between any Blackboard / WEBCT™ data and student 
outcomes. Tables 4 and 5 below list the correlation matrices for Professor Alpha and 
Professor Beta courses. 

Table 4. Correlation Matrix of WEBCT™ variables for Professor Alpha Courses 
COM 2610 (n =14) and COM 3670 (n = 16). 





HITS 


READ 


POSTED 


MAIL 


GRADE 


















(letter) 




2610 


3670 


2610 


3670 


2610 


3670 


2610 3670 


2610 3670 


HITS 


















READ 


.934** 


797 ** 














POSTED 


.816** 


.626** 


.837** 


.514* 










MAIL 


.829** 


.788** 


.657* 


.394 


.605* 


.394 






GRADE 

(letter) 


.548* 


.675** 


.753** 


.501 


.501 


.472 


.776** .281 





** Correlation is significant at the 0.01 level (two-tailed). 
* Correlation is significant at the 0.05 level (two-tailed). 



Table 5. Correlation Matrix of WEBCT™ variables for Professor Beta Courses 
COM 2460 (n = 16) and COM 3470 (n = 17). 

HITS READ POSTED MAIL GRADE 



(letter) 





2460 


3470 


2460 


3470 


2460 


3470 


2460 


3470 2460 


3470 


HITS 




















READ 


.857** 


.741 ** 
















POSTED 


.924** 


.768** 


.762** 


.450 












MAIL 


.916** 


.756** 


.641** 


.270 


.922** 


77 1** 








GRADE 

(letter) 


.732** 


.255 


.750** 


-.072 


.602* 


.439 


.597* 


.407 





** Correlation is significant at the 0.01 level (two-tailed). 
* Correlation is significant at the 0.05 level (two-tailed). 



Apparent from these matrices, a majority of the data are significantly and strongly 
correlated. Specifically for Professor Alpha’s classes, HITS are strongly and significantly 
correlated with READ, POSTED, MAIL, and GRADES across both classes. 

Additionally, READ data correlate strongly and significantly for the lower level course 
with POSTED, MAIL, and GRADES. However, the data that lack significant or strong 
correlations pose the most interesting information. For the upper division class that 
Professor Alpha taught, no significant correlations exist between READ, MAIL, 
GRADES, and POSTED data. The correlation matrix for Professor Beta’s classes offer 
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similar strong and significant correlation as noted for Professor Alpha, while the upper 
level course relationships fail when comparing similar pieces of data. Specifically, for 
Professor Beta’s upper level course POSTED, MAIL, and GRADES fail to correlate 
significantly. 

Examining these matrices for the strongest and most significant relationships 
between student outcomes (grades) and other data, we find that for Professor Alpha’s 
classes, number of hits across both courses correlates most strongly with grades. For the 
lower level course, the number of articles read and the number of postings to the 
discussion forums correlate strongly and significantly with grades. However, for the 
upper level course, a marked lack of significant correlations exists between these data and 
grades. For Professor Beta’s courses, the strongest and most significant correlations exist 
between HITS, READ, POSTED, and MAIL. However, a consistent correlation between 
grades and other data is lacking when considering both courses. The lower- level course 
data for HITS, READ, POSTED, and MAIL correlate strongly and significantly with 
grades, while the data for the upper-level course clearly do not suggest this relationship. 
The discussion suggests how these relationships might be interpreted and how instructors 
might use this information. 

Study 2. Shorter course terms (eight weeks), different course content, same instructor 

Professor Pi taught two eight- week courses that covered media material for 
sophomore and junior students. Course enrollments for both courses totaled 15 students. 
Blackboard / WEBCT™ data (as noted for the previous study) were gathered for both 
shortened courses when the courses concluded. Descriptive data for the two-course 
comparisons are provided in Table 6 below. 

Table 6. Descriptive Statistics of Blackboard / WEBCT ™ variables for Professor Pi eight-week Courses 
CRS 1 and 2 (n = 15 for each course). 





Minimum 


Maximum 


Mean 


SD 




CRS 1 


CRS 2 


CRS 1 


CRS 2 


CRS 1 


CRS 2 


CRS 1 


CRS 2 


HITS 


42 


0 


404 


528 


234.87 


252.93 


135.75 


154.87 


READ 


0 


0 


92 


72 


49.73 


41.60 


39.02 


23.47 


POSTED 


1 


0 


17 


11 


5.67 


4.73 


5.56 


3.77 


MAIL 


0 


0 


33 


44 


10.20 


20.20 


9.63 


15.58 


FOLLUP 


0 


0 


13 


5 


3.20 


2.13 


4.36 


1.60 


GRADE 


0 


0 


4 


4 


2.47 


2.86 


1.64 


1.66 



Apparent from these data, students in CRS 1 engaged in the course more frequently to 
read material, though students in CRS 2 appeared to engage more in the course overall by 
logging in more frequently and sending more personal email. Student outcomes (grades) 
were quite similar for both courses as noted by the means and standard deviations. Thus, 
the measures of central tendency appear to tell little about the engagement levels and any 
relationship to student outcomes. 

The second phase of statistical analysis involved examining the correlation matrix 
for both courses. The question driving this study sought to uncover significant 
relationships (if any) between any WEBCT™ data and student outcomes. Tables 7 and 8 
below list the correlations plus significance levels for CRS 1 and CRS 2 for Professor Pi. 

Table 7. Correlation Matrix of Blackboard / WEBCT™ variables for Professor Pi eight-week 



CRS land CRS 2 (n = 15 for each course). 



HITS READ 


POSTED 


MAIL 


GRADE (ltr) 


CRS 1 CRS 2 CRS 1 CRS 2 


CRS 1 CRS 2 


CRS CRS 
1 2 


CRS 1 CRS 
2 
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HITS 
















READ 


.894** 


.822** 












POSTED 




.789** 


.788** 


.788** 








MAIL 


.737** 


.754** 


.512 


.568* 


.369 


.695** 




GRADE 

(letter) 


.788** 


.700** 


.858** 


.592* 


.722** 


.571* .491 


.409 



** Correlation is significant at the 0.01 level (two-tailed). 
* Correlation is significant at the 0.05 level (two-tailed). 



Examining these data shows a majority of strong correlations with significance levels. 
However, the data that lack significant or strong correlations pose once again the most 
interesting information. MAIL demonstrates neither a strong correlation nor significance 
compared to student outcomes (GRADE). As well, since CRS 1 shared fewer personal 
email interactions, MAIL does not correlate significantly with READ, though CRS 2 
does demonstrate both the relationship between these two variables as well as 
significance. 



Study 3. Same Course Content , Same Instructor and Same Semester 

For this study, two sections of the same course were examined when taught by the 
same instructor during the same semester lasting a full 16 weeks. Blackboard /WEBCT™ 
data were gathered for both courses at the conclusion of the semester. The course content 
provides both an entry-level course for majors in this field as well as serving multiple 
technical disciplines with the college as a required course for their majors. Professor 
Theta had nine (9) students enrolled in one section and 16 students enrolled in the other 
section. Descriptive data for both courses are offered in Table 9 below. 

Table 9. Descriptive Statistics of Blackboard / WEBCT ™ variables for Professor Theta’s courses 
(Course A (n = 9) and B(n = 16)). 





Minimum 


Maximum 


Mean 


SD 




CRS A 


CRS B 


CRS A 


CRS B 


CRS A 


CRS B 


CRS A 


CRS B 


HITS 


45 


114 


773 


506 


322.06 


282.33 


220.57 


106.02 


READ 


0 


3 


133 


123 


51.63 


62.56 


44.57 


42.09 


POSTED 


0 


1 


20 


16 


7.06 


6.78 


6.60 


4.12 


MAIL 


0 


9 


106 


24 


29.81 


17.11 


28.53 


5.13 


FOLLUP 


0 


0 


20 


5 


3.19 


2.56 


5.06 


1.94 


GRADE 


0 


0 


4 


4 


2.63 


2.11 


1.50 


1.69 



These data present a different “picture” compared to the previous two studies. The 
constants in these two courses are the professor and the course material. Both courses 
also are full-semester courses so length of engagement was identical and simultaneous for 
these offerings. However, the student enrollment differed both in numbers and 
individuals, creating different data. The lower enrolled section (CRS A) appears more 
engaged from the start (HITS range) and more consistently engaged (HITS SD) when 
compared to the higher enrolled section (CRS B). Grades spread across the same range 
for each section, but the mean was higher with less deviation in the more-populated 
section. The second phase of analysis mirrors these differences as well. 

The second phase of statistical analysis involved examining the correlation 
matrices for both courses. The question again remains - are Blackboard / WEBCT™ data 
significantly and strongly related to student outcomes (grades)? Table 10 lists the 
correlations for the examined variables for each course. 

Table 10. Correlation Matrix of Blackboard / WEBCT ™ variables for Professor Theta ’s courses 
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CRS A (n = 9) and CRS B (n =16). 





HITS 


READ 


POSTED 


MAIL 


GRADE (ltr) 




CRS A 


CRS B 


CRS A 


CRS B 


CRS A 


CRS B 


CRS 


CRS 


CRS A CRS 
















A 


B 


B 


HITS 




















READ 


.961** 


.747* 
















POSTED 


.724** 


.267 


.769** 


-.116 












MAIL 


.942** 


.120 


.879** 


-.013 


.633** 


.374 








GRADE 

(letter) 


.021 


.325 


.052 


.220 


.016 


.094 


-.030 


-.505 





** Correlation is significant at the 0.01 level (two-tailed). 
* Correlation is significant at the 0.05 level (two-tailed). 



Strong and significant relationships exist for students in CRS A with the following 
Blackboard /WEBCT™ data: HITS, POSTED, MAIL, and READ. However, no 
significant or strong relationships exist between these variables and student outcomes 
(grades). Examining the strength of the relationships between HITS, POSTED, MAIL, 
and READ suggest that students were engaged in this course interpersonally via email 
more so than in course content and course discussions. CRS B students appear to be 
reading the material as evidenced by the HITS significant relationship with READ, but 
grades are not significantly related to any of the Blackboard / WEBCT™ data collected 
for the course. This last study exploring the meaning of Blackboard / WEBCT™ data and 
student outcomes prompts more questions than answers. 

Cluster Analyses Results for All Studies 

Cluster analyses examine the possibility of groups within these courses that share 
structures in the Blackboard / WEBCT™ data without explaining why these structures 
exist. The goal in looking at possible clusters within each course is to determine 
similarities and dissimilarities in the Blackboard / WEBCT™ data. This procedure 
augments the correlational analyses by separating individual cases into groups and then 
examining all Blackboard / WEBCT™ data for each case. When exploring course data in 
this manner, the homogeneity or heterogeneity of the course members affects the results. 
See Table 1 1 below for cluster data. 

For Study 1, hierarchical cluster analyses for Professor Alpha and Beta courses 
produced different results. For Professor Alpha’s courses (2610 and 3670), three distinct 
clusters emerged in the dendograms in the first analysis step. The linkages between 
individuals within the three clusters were similar, and the membership in each cluster was 
somewhat evenly distributed. For the introductory course (2610) though, the membership 
in the “top cluster” (best students) was smaller than membership in this cluster for the 
advanced level course (3670). 

For Professor Beta courses, the dendograms again appeared to create three 
clusters, but in each compared course this time, one individual made up the “top cluster” 
with outlying values on all Blackboard / WEBCT™ data - HITS, READ, POSTED, 
MAIL, and GRDPRCT. The lack of cluster membership and the decreased distances 
between clusters prompted a suggested solution of two clusters for the quick cluster 
analysis. 

Table 11. Quick Cluster Data for All Three Studies Showing Membership and Euclidean Distances 
between Cluster Centers 

Study/Course # of Clusters with Distance between 

membership (X) cluster centers 
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Study 1 

COM2610 01=14) 


1 (2); 2(6); 3(6) 


301.842 

723.105 

1013.909 


COM3670 (n= 1 6) 


1 (4); 2(7); 3(5) 


253.498 

327.479 

578.153 


COM 2460 (n=16) 


1(9); 2(1); 3(6) 


310.793 

531.553 

837.663 


COM3460 (11=17) 


1(8); 2(1); 3(8) 


607.284 

766.182 

1339.396 


Study 2 

CRS 1 


1(6); 2(3); 3(5) 


180.431 

209.769 

390.012 


CRS 2 


1(4); 2(4); 3(7) 


133.357 

184.454 

317.669 


Study 3 

CRS A 


1(5); 2(3); 3(8) 


208.785 

411.714 

620.244 


CRS B 


1(7); 2(2) 


177.255 



For both professors’ course comparisons using quick cluster analysis, individual 
cluster membership focused on the best group of students compared to less competent 
students in the remaining clusters. Professor Alpha had good students, adequate students, 
and less adequate students in the three clusters for each course. Professor Beta had good 
students and average students in the two clusters that emerged in quick cluster analysis. 

In all courses, the cluster membership was distinct and separated by adequate Euclidean 
distances between cluster centers. The cluster centers were determined in a hierarchical 
analysis using the Blackboard / WEBCT™ data in this order: HITS, READ, POSTED, 
MAIL, and finally GRDPRCT. 

For Professor Pi’s shorter term eight-week courses, hierarchical cluster analyses 
produced three clusters in the dendograms with distinct distances ranging from a low of 
133.357 to a high of 390.012 between all three clusters generated for both courses, 
though cluster membership was highest (seven students out of 15) for the “top cluster” 
(best students) in one course. 

For Professor Theta’s sections of the same course (introductory level material), 
hierarchical cluster analyses produced three clusters (n = 16) and two clusters (n = 9). For 
the larger enrolled class, the Euclidean distances between cluster centers were distinct 
(620.244, 411.714, and 208.785) suggesting that the groups were structured differently 
according to the Blackboard / WEBCT™ data. However, the grades for the “top group” 
and “better group” were less disparate (3.33 compared to 3.25) than all other Blackboard 
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/ WEBCT™ data. The smaller enrolled class demonstrated much lower Euclidean 
distances between final cluster centers (177.255) suggesting that the structures in the 
clusters were less distinct, though certainly distinct in grades as the final outcome (1.71 
compared to 3.50 on a 4.00 scale). 



DISCUSSION 

These studies began with four questions. The first question focused on possible 
significant relationships between Blackboard / WEBCT™ data and student grades. The 
obvious answer would assume that the more HITS a student has, the more impact on the 
final grade. However, several exceptions to this hypothesis exist. First, in Study 1, HITS 
did significantly and strongly correlate for both courses. However, articles read in the 
course, number of postings to discussions, and MAIL did NOT correlate significantly 
with the final grades for one course in particular - the upper level course for advanced 
majors. A logical inference here would be that backchanneling or “whispering in the back 
of the classroom” is occurring, detracting from the course content though interaction is 
clearly occurring. 

Second, in this same study, HITS again appeared to relate significantly to final 
grades, but NOT for the upper division course for advanced majors again. Final grades 
were not related to any of the collected Blackboard / WEBCT™ data for the advanced 
course. Once again, a logical inference would suggest that “whispering” among 
classmates contributes to the HITS count, but content likely consists of off-topic material 
or personal communication which does not contribute to a student’s grade. 

In Study 2 with the shorter term courses taught by the same instructor with 
identical enrollments, final grades related significantly and strongly with every variable 
except MAIL, suggesting that interpersonal “whispering” focused on content not related 
to the course. And in the final study, Study 3, no Blackboard / WEBCT™ data related 
significantly to final grades for either course taught by Professor Theta, giving rise to the 
question - what actually determined the students’ final grades in this course? 

Interviews with both instructors in Study 1 elicited qualitative data that deepened 
the view for these courses’ final grades. In both upper-level courses, a mutiny of sorts 
occurred that prompted a dramatic rise in MAIL and backchannel engagement - 
“whispering in the back of the room”. As the MAIL events rose, communication about 
course content may not have risen. Thus, final grades were not related to number of HITS 
as MAIL is included in the number of HITS recorded by Blackboard / WEBCT™. The 
backchanneling is surmised by the instructor to have begun in midcourse when one 
student posted the following message to the public forum: 

I cannot get any help from our instructor. She feels that it’s rude of me 
To bother her on a Saturday night when she has company although she also feels 
it is acceptable to ignore an email I sent on Friday . . . with a question on the 
assignment. Then she blames me for waiting to ask my question til Friday. 

So, I am asking all of you for help. What the hell are we supposed to be basing the 
coal article report on? Is it the newspaper article? Is it the government reports? Do 
we analyze all of them? What??????? 

Failing to get a response from the group beyond one public response, students appeared 
to backchannel and email each other with further comments that may or may not have 
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dealt with course material, as detected by rising MAIL counts immediately following this 
posting. The second instructor teaching both a lower and upper level course had similar 
experiences with bold, critical and public postings eliciting little response while promptly 
thereafter MAIL counts rose dramatically. Thus, without reviewing the content of MAIL 
exchanges for the upper-level courses, one could surmised that these interpersonal 
exchanges focused on other than course content and may have detracted from a focus on 
course material so strongly that as MAIL counts rose for some students in this 
backchanneling, final grades may have suffered. 

For Study 2 with the shorter term courses and similar enrollment figures, final 
grades are strongly and significantly related to all Blackboard / WEBCT™ data except 
MAIL. This suggests again that MAIL exchanges are more likely to be interpersonal and 
not course-related. Study 3 poses the most difficulty in hypothesizing the value of MAIL 
and its influence on final grades since for this study, no data were correlated to final 
grades. Additionally enrollments varied in both of Professor Theta’s classes making even 
the cluster analyses sketchy at best. 

The second question driving these studies focused on productive or destructive 
engagement as a possible tentative inference. Given the qualitative data for the first 
study’s upper level courses, a logical hypothesis would be that MAIL counts are off-topic 
(or non-course related in content) if these data do not significantly relate to final grades. 
Though this relationship can only be tested (or assessed) upon course completion, this 
hypothesis bears further examination across more courses and different disciplines. 

The third question seeks a relationship between Blackboard / WEBCT™ data and 
final grades in online courses. Without Study 3, we might suggest that HITS and READ 
form the strongest relationships with final grades. Again, this hypothesis bears further 
testing. 

Finally, are relational hypotheses possible based upon these studies? Given the 
data from Study 3, the answer to this question would have to be NO — not without further 
examination of other variables. In each study and each online course, a number of 
human-related variables are not considered here - professor’s communication style 
teaching online, professor’s experience and enjoyment in teaching online, number of 
students in a course, previous relationships between professors and students, students’ 
experience in online learning environments, students’ levels of digital literacy overall, 
course material (technical, abstract, or concrete), students’ preparation and motivation 
regarding course material, and type of assignments contributing to the final grade 
(subjective versus objective or a combination). Additionally, students in an online course 
may have different course designs waiting for them when they log into a course. Some 
online courses direct students to reading material and research outside the course 
software, while other online courses engage students in substantive material presented 
through the course software or discussion forums. In the former case, HITS and READ 
data may not directly relate to learning the course material, while in the latter these data 
may indeed reflect course content and engagement. 

CONCLUSIONS 

Overall, the data gathered by Blackboard / WEBCT™ courseware in an online 
course offered virtual snapshots of student activities and engagement. The cluster 
analyses for all studies suggest that individual student performance in a course creates 
group clusters based upon the Blackboard / WEBCT™ data as collected. Thus, all the 
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Blackboard / WEBCT™ data taken as a whole and hierarchically — with HITS having the 
strongest impact upon cluster formations - form groups of students that are characterized 
by their course engagement which influences the course grade. When reflecting back 
upon the questions guiding these studies, the answers are mixed at best. 

What significant relationships exist among the Blackboard/Web CT data for an 
online course that reflect student outcomes as defined by final grade? Clearly the data in 
these studies suggest that HITS and READ data plus POSTED data more often have 
significant relationships with final grades. However, one course’s data fail to 
demonstrate any significant correlations with final grades. The question here becomes 
what, if anything online or offline, impacts student final grades? Further studies in this 
area including qualitative methods are required. 

The second question guiding these studies focuses on the possibility of logically 
inferring general communication content in backchanneled (or whispered) interactions 
without viewing the message content. The data support logical inferences regarding 
general message content by examining significant correlations between HITS/MAIL and 
final grades. In courses where the MAIL counts rose but the significant correlations to 
final grades failed to exist, one may infer that message content is either personal or off- 
topic. The qualitative data from interviews with instructors who taught upper level 
advanced majors courses suggested this inference once the lack of significant correlation 
was examined. 

The third question seeks insights into what, if any, collected data in an online 
course shell contribute to student final grades. In all courses where significant 
correlations were found, HITS and READ contributed the strongest significant 
relationships with student final grades. This information suggests that student 
engagement with the course (HITS) and student processing of course materials (READ) 
impact positively a student’s final grade. 

Coupled with content and instructor awareness of interactions, it may be possible 
to suggest tentatively that specific interpersonal communications ala the “tell tale data” 
(student-student MAIL) are off-topic or neutral contributors to the student’s final grade in 
some cases. However, doing something about this type of interaction remains the key to 
keeping an online course on track for substantive engagement and successful student 
outcomes. Only after gathering qualitative data through interviews with instructors is a 
deeper understanding of the course data possible. Clearly further studies with more 
qualitative data may broaden the picture in lieu of relying on snapshots in numerical time. 
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